Spelling Normalization of Historical German with Sparse Training Data

Recently, there has been a growing interest in historical language corpora. Projects to create such corpora exist for a variety of languages such as German (Scheible et al. 2011), Spanish (SánchezMarco et al. 2010), or Slovene (Erjavec 2012). Annotation of these corpora is complicated by the fact that specialized tools for these language stages are typically not available. A common approach is to employ spelling normalization to map historical wordforms to modern ones (e.g., Adesam et al. 2012, Baron et al. 2009, Jurish 2010), so that existing tools for modern language (e.g., modern POS taggers) can be used on the normalized data. This paper presents an approach to spelling normalization that combines three different normalization algorithms and evaluates it on a diverse set of texts of historical German. The evaluation shows that this approach produces acceptable results even with comparatively small amounts of training data. The normalization methods were previously described in Bollmann (2012), though with a much more restricted evaluation.